GRADE guidance 37: rating imprecision in a body of evidence on test accuracy.

OBJECTIVES: To provide guidance on rating imprecision in a body of evidence assessing the accuracy of a single test. This guide will clarify when Grading of Recommendations Assessment, Development and Evaluation (GRADE) users should consider rating down the certainty of evidence by one or more levels for imprecision in test accuracy. STUDY DESIGN AND SETTING: A project group within the GRADE working group conducted iterative discussions and presentations at GRADE working group meetings to produce this guidance. RESULTS: Before rating the certainty of evidence, GRADE users should define the target of their certainty rating. GRADE recommends setting judgment thresholds defining what they consider a very accurate, accurate, inaccurate, and very inaccurate test. These thresholds should be set after considering consequences of testing and effects on people-important outcomes. GRADE's primary criterion for judging imprecision in test accuracy evidence is considering confidence intervals (i.e., CI approach) of absolute test accuracy results (true and false, positive, and negative results in a cohort of people). Based on the CI approach, when a CI appreciably crosses the predefined judgment threshold(s), one should consider rating down certainty of evidence by one or more levels, depending on the number of thresholds crossed. When the CI does not cross judgment threshold(s), GRADE suggests considering the sample size for an adequately powered test accuracy review (optimal or review information size [optimal information size (OIS)/review information size (RIS)]) in rating imprecision. If the combined sample size of the included studies in the review is smaller than the required OIS/RIS, one should consider rating down by one or more levels for imprecision. CONCLUSION: This paper extends previous GRADE guidance for rating imprecision in single test accuracy systematic reviews and guidelines, with a focus on the circumstances in which one should consider rating down one or more levels for imprecision.

Patients' values and preferences for health states in allergic rhinitis-An artificial intelligence supported systematic review.

BACKGROUND: Allergic rhinitis (AR) impacts patients' physical and emotional well-being. Assessing patients' values and preferences (V&P) related to AR is an essential part of patient-centered care and of the guideline development process. We aimed to systematically summarize the information about patients' V&P on AR and its symptoms and impact on daily life. METHODS: We conducted systematic review in a MEDLINE, Embase, PsychInfo, and CINAHL databases. We included studies which quantitatively assessed patients' V&P for specific outcomes in AR by assessing utilities, applying discrete choice approaches, or rating and ranking outcomes. We grouped outcomes as AR symptoms, functional status, and care-related patient experience. Study selection and data extraction were supported by the Laser AI tool. We rated the certainty of evidence (CoE) using the GRADE approach. RESULTS: Thirty-six studies (41 records) were included: nine utility studies, seven direct-choice studies and 21 studies of rating or ranking outcomes. Utilities were lower with increased AR severity and with the concomitant presence of asthma, but not with whether AR was seasonal or perennial (CoE = low-high). Patients rated AR symptom-related outcomes as more important than those related to care-related patient experience and functional status (CoE = very low-moderate). Nasal symptoms (mainly nasal congestion) followed by breathing disorders, general and ocular symptoms were rated as the symptoms with the highest impact. CONCLUSIONS: This systematic review provides a comprehensive overview of V&P of patients with AR. Patients generally considered nasal symptoms as the most important. Future studies with standardized methods are needed to provide more information on V&P in AR.

A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).

BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.

User-experience testing of an evidence-to-decision framework for selecting essential medicines.

Essential medicine lists (EMLs) are important medicine prioritization tools used by the World Health Organization (WHO) EML and over 130 countries. The criteria used by WHO's Expert Committee on the Selection and Use of Essential Medicines has parallels to the GRADE Evidence-to-Decision (EtD) frameworks. In this study, we explored the EtD frameworks and a visual abstract as adjunctive tools to strengthen the integrate evidence and improve the transparency of decisions of EML applications. We conducted user-experience testing interviews of key EML stakeholders using Morville's honeycomb model. Interviews explored multifaceted dimensions (e.g., usability) on two EML applications for the 2021 WHO EML-long-acting insulin analogues for diabetes and immune checkpoint inhibitors for lung cancer. Using a pre-determined coding framework and thematic analysis we iteratively improved both the EtD framework and the visual abstract. We coded the transcripts of 17 interviews with 13 respondents in 103 locations of the interview texts across all dimensions of the user-experience honeycomb. Respondents felt the EtD framework and visual abstract presented complementary useful and findable adjuncts to the traditional EML application. They felt this would increase transparency and efficiency in evidence assessed by EML committees. As EtD frameworks are also used in health practice guidelines, including those by the WHO, respondents articulated that the adoption of the EtD by EML applications represents a tangible mechanism to align EMLs and guidelines, decrease duplication of work and improve coordination. Improvements were made to clarify instructions for the EtD and visual abstract, and to refine the design and content included. 'Availability' was added as an additional criterion for EML applications to highlight this criterion in alignment with WHO EML criteria. EtD frameworks and visual abstracts present additional important tools to communicate evidence and support decision-criteria in EML applications, which have global health impact. Access to essential medicines is important for achieving universal health coverage, and the development of essential medicine lists should be as evidence-based and trustworthy as possible.

Operationalizing the GRADE-equity criterion to inform guideline recommendations: application to a medical cannabis guideline.

OBJECTIVES: Incorporating health equity considerations into guideline development often requires information beyond that gathered through traditional evidence synthesis methodology. This article outlines an operationalization plan for the Grading of Recommendations Assessment, Development, and Evaluation (GRADE)-equity criterion to gather and assess evidence from primary studies within systematic reviews, enhancing guideline recommendations to promote equity. We demonstrate its use in a clinical guideline on medical cannabis for chronic pain. STUDY DESIGN AND SETTING: We reviewed GRADE guidance and resources recommended by team members regarding the use of evidence for equity considerations, drafted an operationalization plan, and iteratively refined it through team discussion and feedback and piloted it on a medicinal cannabis guideline. RESULTS: We propose a seven-step approach: 1) identify disadvantaged populations, 2) examine available data for specific populations, 3) evaluate population baseline risk for primary outcomes, 4) assess representation of these populations in primary studies, 5) appraise analyses, 6) note barriers to implementation of effective interventions for these populations, and 7) suggest supportive strategies to facilitate implementation of effective interventions. CONCLUSION: Our approach assists guideline developers in recognizing equity considerations, particularly in resource-constrained settings. Its application across various guideline topics can verify its feasibility and necessary adjustments.

A multimethods randomized trial found that plain language versions improved adults understanding of health recommendations.

OBJECTIVES: To make informed decisions, the general population should have access to accessible and understandable health recommendations. To compare understanding, accessibility, usability, satisfaction, intention to implement, and preference of adults provided with a digital "Plain Language Recommendation" (PLR) format vs. the original "Standard Language Version" (SLV). STUDY DESIGN AND SETTING: An allocation-concealed, blinded, controlled superiority trial and a qualitative study to understand participant preferences. An international on-line survey. 488 adults with some English proficiency. 67.8% of participants identified as female, 62.3% were from the Americas, 70.1% identified as white, 32.2% had a bachelor's degree as their highest completed education, and 42% said they were very comfortable reading health information. In collaboration with patient partners, advisors, and the Cochrane Consumer Network, we developed a plain language format of guideline recommendations (PLRs) to compare their effectiveness vs. the original standard language versions (SLVs) as published in the source guideline. We selected two recommendations about COVID-19 vaccine, similar in their content, to compare our versions, one from the World Health Organization (WHO) and one from Centers for Disease Control and Prevention (CDC). The primary outcome was understanding, measured as the proportion of correct responses to seven comprehension questions. Secondary outcomes were accessibility, usability, satisfaction, preference, and intended behavior, measured on a 1-7 scale. RESULTS: Participants randomized to the PLR group had a higher proportion of correct responses to the understanding questions for the WHO recommendation (mean difference [MD] of 19.8%, 95% confidence interval [CI] 14.7-24.9%; P < 0.001) but this difference was smaller and not statistically significant for the CDC recommendation (MD of 3.9%, 95% CI -0.7% to 8.3%; P = 0.096). However, regardless of the recommendation, participants found the PLRs more accessible, (MD of 1.2 on the seven-point scale, 95% CI 0.9-1.4%; P < 0.001) and more satisfying (MD of 1.2, 95% CI 0.9-1.4%; P < 0.001). They were also more likely to follow the recommendation if they had not already followed it (MD of 1.2, 95% CI 0.7-1.8%; P < 0.001) and share it with other people they know (MD of 1.9, 95% CI 0.5-1.2%; P < 0.001). There was no significant difference in the preference between the two formats (MD of -0.3, 95% CI -0.5% to 0.03%; P = 0.078). The qualitative interviews supported and contextualized these findings. CONCLUSION: Health information provided in a PLR format improved understanding, accessibility, usability, and satisfaction and thereby has the potential to shape public decision-making behavior.

GRADE Concept 7: Issues and Insights Linking Guideline Recommendations to Trustworthy Essential Medicine Lists.

OBJECTIVES: Guidelines and essential medicine lists (EMLs) bear similarities and differences in the process that lead to decisions. Access to essential medicines is central to achieve universal health coverage. The World Health Organization (WHO) EML has guided prioritization of essential medicines globally for nearly 50 years, and national EMLs (NEMLs) exist in over 130 countries. Guideline and EML decisions, at WHO or national levels, are not always coordinated and aligned. We sought to explore challenges, and potential solutions, for decision-making to support trustworthy medicine selection for EMLs from a Grading of Recommendations, Assessment, Development and Evaluations (GRADE) Working Group perspective. We primarily focus on the WHO EML; however, our findings may be applicable to NEML decisions as well. STUDY DESIGN AND SETTING: We identified key challenges in connecting the EML to health guidelines by involving a broad group of stakeholders and assessing case studies including real applications to the WHO EML, South Africa NEML, and a multiple sclerosis guideline connected to a WHO EML application for multiple sclerosis treatments. To address challenges, we utilized the results of a survey and feedback from the stakeholders, and iteratively met as a project group. We drafted a conceptual framework of challenges and potential solutions. We presented a summary of the results for feedback to all attendees of the GRADE Working Group meetings in November 2022 (approximately 120 people) and in May 2023 (approximately 100 people) before finalizing the framework. RESULTS: We prioritized issues and insights/solutions that addressed the connections between the EML and health guidelines. Our suggested solutions include early planning alignment of guideline groups and EMLs, considering shared participation to strengthen linkage, further clarity on price/cost considerations, and using explicit shared criteria to make guideline and EML decisions. We also provide recommendations to strengthen the connection between WHO EML and NEMLs including through contextualization methods. CONCLUSION: This GRADE concept article, jointly developed by key stakeholders from the guidelines and EMLs field, identified key conceptual issues and potential solutions to support the continued advancement of trustworthy EMLs. Adopting structured decision criteria that can be linked to guideline recommendations bears the potential to advance health equity and gaps in availability of essential medicines within and between countries.

Intranasal antihistamines and corticosteroids in the treatment of allergic rhinitis: a systematic review and meta-analysis protocol.

INTRODUCTION: Intranasal antihistamines and corticosteroids are some of the most frequently used drug classes in the treatment of allergic rhinitis. However, there is uncertainty as to whether effectiveness differences may exist among different intranasal specific medications. This systematic review aims to analyse and synthesise all evidence from randomised controlled trials (RCTs) on the effectiveness of intranasal antihistamines and corticosteroids in rhinitis nasal and ocular symptoms and in rhinoconjunctivitis-related quality-of-life. METHODS AND ANALYSIS: We will search four electronic bibliographic databases and three clinical trials databases for RCTs (1) assessing patients ≥12 years old with seasonal or perennial allergic rhinitis and (2) comparing the use of intranasal antihistamines or corticosteroids versus placebo. Assessed outcomes will include the Total Nasal Symptom Score (TNSS), the Total Ocular Symptom Score (TOSS) and the Rhinoconjunctivitis Quality-of-Life Questionnaire (RQLQ). We will assess the methodological quality of included primary studies by using the Cochrane risk-of-bias tool. Certainty in the body of evidence for the analysed outcomes will be assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. We will perform a random-effects meta-analysis for each assessed medication and outcome, presenting results as pooled mean differences and standardised mean differences. Heterogeneity will be explored by sensitivity and subgroup analyses, considering (1) the risk of bias, (2) the follow-up period and (3) the drug dose. ETHICS AND DISSEMINATION: Ethical considerations will not be required. Results will be disseminated in a peer-review journal. PROSPERO REGISTRATION NUMBER: CRD42023416573.

Antibiotic prescribing guideline recommendations in COVID-19: a systematic survey.

Background: COVID-19 and antimicrobial resistance (AMR) are two intersecting public health crises. Antimicrobial overuse in patients with COVID-19 threatens to worsen AMR. Guidelines are fundamental in encouraging antimicrobial stewardship. We sought to assess the quality of antibiotic prescribing guidelines and recommendations in the context of COVID-19, and whether they incorporate principles of antimicrobial stewardship. Methods: We performed a systematic survey which included a search using the concepts "antibiotic/antimicrobial" up to November 15, 2022 of the eCOVID-19 living map of recommendations (RecMap) which aggregates guidelines across a range of international sources and all languages. Guidelines providing explicit recommendations regarding antibacterial use in COVID-19 were eligible for inclusion. Guideline and recommendation quality were assessed using the AGREE II and AGREE-REX instruments, respectively. We extracted guideline characteristics including panel representation and the presence or absence of explicit statements related to antimicrobial stewardship (i.e., judicious antibiotic use, antimicrobial resistance or adverse effects as a consequence of antibiotic use). We used logistic regression to evaluate the relationship between guideline characteristics including quality and incorporation of antimicrobial stewardship principles. Protocol registration (OSF): https://osf.io/4pgtc. Findings: Twenty-eight guidelines with 63 antibiotic prescribing recommendations were included. Recommendations focused on antibiotic initiation (n = 52, 83%) and less commonly antibiotic selection (n = 13, 21%), and duration of therapy (n = 15, 24%). Guideline and recommendation quality varied widely. Twenty (71%) guidelines incorporated at least one concept relating to antimicrobial stewardship. Including infectious diseases expertise on the guideline panel (OR 9.44, 97.5% CI: 1.09-81.59) and AGREE-REX score (OR 3.26, 97.5% CI: 1.14-9.31 per 10% increase in overall score) were associated with a higher odds of guidelines addressing antimicrobial stewardship. Interpretation: There is an opportunity to improve antibiotic prescribing guidelines in terms of both quality and incorporation of antimicrobial stewardship principles. These findings can help guideline developers better address antibiotic stewardship in future recommendations beyond COVID-19. Funding: This project was funded by Michael G. DeGroote Cochrane Canada and McMaster GRADE centres.

GRADE concept 6: a novel application of external indirect evidence into GRADE ratings of evidence certainty in network meta-analysis.

OBJECTIVES: We describe how consideration of external evidence may play an important role in judging certainty in the process of establishing the certainty of the evidence. Our example is a network meta-analysis (NMA) addressing treatment for Ebola virus disease, which informed a World Health Organization guideline. STUDY DESIGN AND SETTING: Through Grading of Recommendations Assessment, Development, and Evaluations (GRADE) project group iterative online, in-person and email discussions, we developed this GRADE concept and obtained approval from the GRADE working group. Using the null as a threshold, we rated our certainty for network estimates in mortality, including consideration of evidence external to the NMA (i.e., did not meet eligibility criteria) and formal logical construction. RESULTS: Based on the existing GRADE guidance, we rated the network estimate for one indirect comparison as low certainty. The formal logical construction that lead us reevaluate the certainty of the evidence is as follows: if A is superior to B, and B is not inferior to C, then A must be superior to C. After considering the logic and the external indirect evidence, we concluded at least moderate certainty for the comparison. CONCLUSION: Systematic review authors and guideline developers should apply the fundamental logical construction for indirect comparisons and consider compelling external evidence in NMA certainty ratings.

GRADE GUIDANCE 38: Updated guidance for rating up certainty of evidence due to a dose-response gradient.

INTRODUCTION: This updated guidance from the Grading of Recommendations Assessment, Development, and Evaluation addresses rating up certainty of evidence due to a dose-response gradient (DRG) observed in synthesis of intervention and exposure studies. STUDY DESIGN AND SETTING: This guidance was developed using iterative discussions and consensus in multiple meetings and was presented to attendees of the Grading of Recommendations Assessment, Development, and Evaluation Working Group meeting for feedback in November 2022 and for final approval in May 2023. RESULTS: The guidance consists of two steps. The first is to determine whether the DRG is credible. We describe five items for assessing credibility: a) is DRG identified using a proper analytical approach; b) is confounding the cause of the DRG; c) is there serious concern about ecological bias; d) is the DRG consistent across studies; and e) is there indirect evidence supporting the DRG. The first two of these items are the most critical. If the DRG was judged to be credible, then the second step is to apply the DRG domain and consider rating up, but only by one level due to the concern about residual confounding. CONCLUSION: Systematic review authors should only rate up certainty in evidence when a DRG is deemed credible.

GRADE guidance 23: considering cost-effectiveness evidence in moving from evidence to health-related recommendations.

BACKGROUND: This is the 23rd in a series of articles describing the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to grading the certainty of evidence and strength of recommendations for systematic reviews, health technology assessments, and clinical guideline development. OBJECTIVES: We outline how resource utilization and cost-effectiveness analyses are integrated into health-related recommendations, using the GRADE Evidence to Decision (EtD) frameworks. STUDY DESIGN AND SETTING: Through iterative discussions and refinement, in-person, and online meetings, and through e-mail communication, we developed draft guidance to incorporate economic evidence in the formulation of health-related recommendations. We developed scenarios to operationalize the guidance. We presented a summary of the results to members of the GRADE Economic Evaluation Project Group. RESULTS: We describe how to estimate the cost of preventing (or achieving) an event to inform assessments of cost-effectiveness of alternative treatments, when there are no published economic evaluations. Evidence profiles and Summary of Findings tables based on systematic reviews of cost-effectiveness analyses can be created to provide top-level summaries of results and quality of multiple published economic evaluations. We also describe how this information could be integrated in GRADE's EtD frameworks to inform health-related recommendations. Three scenarios representing various levels of available cost-effectiveness evidence were used to illustrate the integration process. CONCLUSION: This GRADE guidance provides practical information for presenting cost-effectiveness data and its integration in the development of health-related recommendations, using the EtD frameworks.

Plain Language vs Standard Format for Youth Understanding of COVID-19 Recommendations: A Randomized Clinical Trial.

Importance: To ensure that youths can make informed decisions about their health, it is important that health recommendations be presented for understanding by youths. Objective: To compare understanding, accessibility, usability, satisfaction, intention to implement, and preference of youths provided with a digital plain language recommendation (PLR) format vs the original standard language version (SLV) of a health recommendation. Design, Setting, and Participants: This pragmatic, allocation-concealed, blinded, superiority randomized clinical trial included individuals from any country who were 15 to 24 years of age, had internet access, and could read and understand English. The trial was conducted from May 27 to July 6, 2022, and included a qualitative component. Interventions: An online platform was used to randomize youths in a 1:1 ratio to an optimized digital PLR or SLV format of 1 of 2 health recommendations related to the COVID-19 vaccine; youth-friendly PLRs were developed in collaboration with youth partners and advisors. Main Outcomes and Measures: The primary outcome was understanding, measured as the proportion of correct responses to 7 comprehension questions. Secondary outcomes were accessibility, usability, satisfaction, preference, and intended behavior. After completion of the survey, participants indicated their interest in completing a 1-on-1 semistructured interview to reflect on their preferred digital format (PLR or SLV) and their outcome assessment survey response. Results: Of the 268 participants included in the final analysis, 137 were in the PLR group (48.4% female) and 131 were in the SLV group (53.4% female). Most participants (233 [86.9%]) were from North and South America. No significant difference was found in understanding scores between the PLR and SLV groups (mean difference, 5.2%; 95% CI, -1.2% to 11.6%; P = .11). Participants found the PLR to be more accessible and usable (mean difference, 0.34; 95% CI, 0.05-0.63) and satisfying (mean difference, 0.39; 95% CI, 0.06-0.73) and had a stronger preference toward the PLR (mean difference, 4.8; 95% CI, 4.5-5.1 [4.0 indicated a neutral response]) compared with the SLV. No significant difference was found in intended behavior (mean difference, 0.22 (95% CI, -0.20 to 0.74). Interviewees (n = 14) agreed that the PLR was easier to understand and generated constructive feedback to further improve the digital PLR. Conclusions and Relevance: In this randomized clinical trial, compared with the SLV, the PLR did not produce statistically significant findings in terms of understanding scores. Youths ranked it higher in terms of accessibility, usability, and satisfaction, suggesting that the PLR may be preferred for communicating health recommendations to youths. The interviews provided suggestions for further improving PLR formats. Trial Registration: ClinicalTrials.gov Identifier: NCT05358990.

A multimethods randomized trial found that plain language versions improved parents' understanding of health recommendations.

OBJECTIVES: To assess the effectiveness of plain language compared with standard language versions of COVID-19 recommendations specific to child health. STUDY DESIGN AND SETTING: Pragmatic, allocation-concealed, blinded, superiority randomized controlled trial with nested qualitative component. Trial was conducted online, internationally. Parents or legal guardians (≥18 years) of a child (<18 years) were eligible. Participants were randomized to receive a plain language recommendation (PLR) or standard (SLV) verison of a COVID-19 recommendation specific to child health. Primary outcome was understanding. Secondary outcomes included: preference, accessibility, usability, satisfaction, and intended behavior. Interviews explored perceptions and preferences for each format. RESULTS: Between July and August 2022, 295 parents were randomized; 241 (81.7%) completed the study (intervention n = 121, control n = 120). Mean understanding scores were significantly different between groups (PLR 3.96 [standard deviation (SD) 2.02], SLV 3.33 [SD 1.88], P = 0.014). Overall participants preferred the PLR version: mean rating 5.05/7.00 (95% CI 4.81, 5.29). Interviews (n = 12 parents) highlighted their preference for the PLR and provided insight on elements to enhance future knowledge mobilization of health recommendations. CONCLUSION: Compared to SLVs, parents preferred PLRs and better understood the recommendation. Guideline developers should strive to use plain language to increase understanding, uptake, and implementation of evidence by the public.

GRADE concept 4: rating the certainty of evidence when study interventions or comparators differ from PICO targets.

OBJECTIVES: This Grading of Recommendations Assessment, Development and Evaluation (GRADE) concept article offers systematic reviewers, guideline authors, and other users of evidence assistance in addressing randomized trial situations in which interventions or comparators differ from those in the target people, interventions, comparators, and outcomes. To clarify what GRADE considers under indirectness of interventions and comparators, we focus on a particular example: when comparator arm participants receive some or all aspects of the intervention management strategy (treatment switching). STUDY DESIGN AND SETTING: An interdisciplinary panel of the GRADE working group members developed this concept article through an iterative review of examples in multiple teleconferences, small group sessions, and e-mail correspondence. After presentation at a GRADE working group meeting in November 2022, attendees approved the final concept paper, which we support with examples from systematic reviews and individual trials. RESULTS: In the presence of safeguards against risk of bias, trials provide unbiased estimates of the effect of an intervention on the people as enrolled, the interventions as implemented, the comparators as implemented, and the outcomes as measured. Within the GRADE framework, differences in the people, interventions, comparators, and outcomes elements between the review or guideline recommendation targets and the trials as implemented constitute issues of indirectness. The intervention or comparator group management strategy as implemented, when it differs from the target comparator, constitutes one potential source of indirectness: Indirectness of interventions and comparators-comparator group receipt of the intervention constitutes a specific subcategory of said indirectness. The proportion of comparator arm participants that received the intervention and the apparent magnitude of effect bear on whether one should rate down, and if one does, to what extent. CONCLUSION: Treatment switching and other differences between review or guideline recommendation target interventions and comparators vs. interventions and comparators as implemented in otherwise relevant trials are best considered issues of indirectness.

American Society of Hematology 2023 guidelines for management of venous thromboembolism: thrombophilia testing.

Hereditary and acquired thrombophilia are risk factors for venous thromboembolism (VTE). Whether testing helps guide management decisions is controversial. These evidence-based guidelines from the American Society of Hematology (ASH) intend to support decision making about thrombophilia testing. ASH formed a multidisciplinary guideline panel covering clinical and methodological expertise and minimizing bias from conflicts of interest. The McMaster University GRADE Centre provided logistical support, performed systematic reviews, and created evidence profiles and evidence-to-decision tables. The Grading of Recommendations Assessment, Development, and Evaluation approach (GRADE) was used. Recommendations were subject to public comment. The panel agreed on 23 recommendations regarding thrombophilia testing and associated management. Nearly all recommendations are based on very low certainty in the evidence due to modeling assumptions. The panel issued a strong recommendation against testing the general population before starting combined oral contraceptives (COCs) and conditional recommendations for thrombophilia testing in the following scenarios: (a) patients with VTE associated with nonsurgical major transient or hormonal risk factors; (b) patients with cerebral or splanchnic venous thrombosis, in settings where anticoagulation would otherwise be discontinued; (c) individuals with a family history of antithrombin, protein C, or protein S deficiency when considering thromboprophylaxis for minor provoking risk factors and for guidance to avoid COCs/hormone replacement therapy; (d) pregnant women with a family history of high-risk thrombophilia types; and (e) patients with cancer at low or intermediate risk of thrombosis and with a family history of VTE. For all other questions, the panel provided conditional recommendations against testing for thrombophilia.

GRADE Reporting in Systematic Reviews Published in the Urological Literature (2009-2021).

PURPOSE: We evaluate the reporting of the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach to rating the certainty of evidence in systematic reviews published in the urological literature. MATERIALS AND METHODS: Based on a predefined protocol, we identified all systematic reviews published in 5 major urological journals from 1998 to 2021 that reported the use of GRADE. Two authors performed study selection and data abstraction independently to assess reporting in accordance with established criteria for applying GRADE. RESULTS: We included 68 of 522 (13.0%) systematic reviews that reported the use of GRADE; the first was published in 2009. Approximately half were published between 2009-2018 (n=36) and the other half between 2019-2021 (n=32). Oncology (24; 35.3%) was the most common clinical topic, and the authors were mostly based in Europe (34; 50%). In their abstract, less than half of all systematic reviews (32; 47.1%) provided any certainty of evidence rating. Only 41 (60.3%) included a tabular result summary in the format of a summary of findings table (24; 35.3%) or evidence profile (17; 25.0%). Few (35.3%) addressed the GRADE certainty of evidence rating in the discussion section. Reporting did not improve over time when comparing the 2 time periods. CONCLUSIONS: Whereas GRADE is increasingly being applied for rating the certainty of evidence, systematic reviews published in the urological literature frequently have not followed established criteria for applying or using GRADE. There is a need for better training of authors and editors, as well as for a GRADE reporting checklist for systematic review authors.